Information processing apparatus and computer-readable recording medium recording information processing program

ABSTRACT

Motion information is obtained which is information about a motion applied to an input device housing itself including a pointing device of a plurality of input mean. Next, based on the motion information, a movement amount of the input device housing is calculated. Thereafter, it is determined whether or not the movement amount satisfies predetermined conditions. When the predetermined conditions are satisfied, a position is designated based on an output from the pointing device.

CROSS REFERENCE TO RELATED APPLICATION

The disclosure of Japanese Patent Application No. 2008-267823, filedOct. 16, 2008, is incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an information processing apparatus anda computer-readable recording medium recording an information processingprogram which are used to designate a position on a screen and execute aselection process. More particularly, the present invention relates toan information processing apparatus and a computer-readable recordingmedium recording an information processing program which are used todesignate a position on a screen and execute a selection process basedon outputs of a plurality of input means including at least a pointingdevice.

2. Description of the Background Art

Conventionally, in a computer system and the like, a coordinate inputapparatus has been used so as to designate process items (e.g., a menu).Among its application forms, a plurality of coordinate inputapparatuses, such as a mouse and a tablet, may be connected to a systemand used in combination as appropriate. When such an application form isemployed, a plurality of coordinate input apparatuses need to beswitched as required. An apparatus for switching these coordinate inputapparatuses is known (e.g., Japanese Patent Laid-Open Publication No.H06-208435). In the coordinate input apparatus switching apparatus, acoordinate input apparatus is selected in which changes in designatedcoordinate continually occur for a predetermined period of time or more.Specifically, one of the mouse and the tablet is selected only when ithas been operated for a predetermined period of time.

However, the aforementioned apparatus for switching coordinate inputapparatuses has the following problem when it switches the coordinateinput apparatuses. Specifically, the switching apparatus selects acoordinate input apparatus to be used, based on whether or not thecoordinate input apparatus has been operated for a predetermined periodof time or more as described above. Therefore, disadvantageously, ittakes a relatively long time to switch coordinate input apparatuses. Inother words, the response to switching is slow. Therefore, coordinateinput apparatuses cannot be switched in quick response at a timingdesired by the user.

SUMMARY OF THE INVENTION

Therefore, an object of the present invention is to provide aninformation processing apparatus and a computer-readable recordingmedium recording an information processing program that can switch aplurality of input means in quick response.

The present invention has the following features to attain the objectmentioned above. Note that reference numerals, additional descriptionsand the like inside parentheses in this section indicate correspondenceto embodiments described below for the sake of easy understanding, anddo not limit the present invention.

A first aspect of the present invention is directed to acomputer-readable recording medium recording an information processingprogram executable by a computer of an information processing apparatusfor designating a position on a screen based on outputs of a pluralityof input means including at least one pointing device, the programcausing the computer to function as: an motion information obtainingmeans (S47), a movement amount calculating means (S48), a thresholddetermining means (S52), and a first position designating means (S54,S24). The motion information obtaining means obtains motion informationthat is information about a motion applied to a first input devicehousing itself including the pointing device. The movement amountcalculating means calculates a movement amount of the first input devicehousing based on the motion information. The threshold determining meansdetermines whether or not the movement amount exceeds a predeterminedthreshold. The first position designating means designates a positionbased or an output from the pointing device when the thresholddetermining means determines that the movement amount exceeds thepredetermined threshold.

According to the first aspect, when a motion applied to an input deviceitself including a pointing device is detected, and based on the motion,then the operation can be switched to a position designating operationthat is performed using the pointing device. Thereby, when a pluralityof input means are used to perform an operation, it is possible toswitch the input means used by the player in quick response.

In a second aspect based on the first aspect, the first input devicehousing further includes a non-pointing device that is an input meansthat is not a pointing device, of the plurality of input means. Theinformation processing program causes the computer to further functionas a determination means (S23) and a second position designating means(S25, S26). The determination means determines whether or not an inputto the non-pointing device has occurred when position designation isbeing performed by the first position designating means. The secondposition designating means designates a position based on an output fromthe non-pointing device when the determination means determines that aninput to the non-pointing device has occurred.

According to the second aspect, when input devices including a pointingdevice and a non-pointing device are used in the same housing, operationmethods of the devices can be switched in quick response.

In a third aspect based on the first aspect, a non-pointing device thatis an input means that is not a pointing device, of the plurality ofinput means, is included in a second input device housing, and theinformation processing program causes the computer to further functionas a determination means (S23) and a second position designating means(S25, S26). The determination means determines whether or not an inputto the non-pointing device has occurred when position designation isbeing performed by the first position designating means. The secondposition designating means designates a position based on an output fromthe non-pointing device when the determination means determines that aninput to the non-pointing device has occurred.

According to the third aspect, when a pointing device and a non-pointingdevice are provided in separate housings, input methods employing thedevices can be switched in quick response.

In a fourth aspect based on the first aspect, the information processingprogram causes the computer to further function as a switching means(S27, S54) for switching a position designation mode between a firstposition designation mode in which position destination is performedusing the first position designating means and a second positiondesignation mode in which position designation is performed based on anoutput from a non-pointing device that is an input means that is not apointing device. The switching means switches the position designationmode to the first position designation mode if the movement amountexceeds a predetermined threshold when the position designation mode isthe second position designation mode, and switches the positiondesignation mode to the second position designation mode if an input tothe non-pointing device has occurred when the position designation modeis the first position designation mode.

According to the fourth aspect, a method of operating a pointing devicefor position designation and a method of operating a non-pointing devicefor position designation can be switched in quick response, depending ona motion applied to the input device housing itself.

In a fifth aspect based on the fourth aspect, the information processingprogram causes the computer to further function as a no-input periodmeasuring means (S46, S53) for measuring a period of time during whichit is determined that there is not an input to the non-pointing devicewhen the position designation mode is the second position designationmode. The switching means switches the position designation mode to thefirst position designation mode when the period of time measured by theno-input period measuring means exceeds a predetermined value.

According to the fifth aspect, the operability of an input device can beimproved, thereby enhancing the convenience for the user.

In a sixth aspect based on the first aspect, the first input devicehousing includes a motion sensor for detecting a motion applied to thefirst input device housing itself. The motion information obtainingmeans obtains data output from the motion sensor as the motioninformation.

According to the sixth aspect, a motion applied to the first inputdevice housing itself can be more accurately recognized, thereby makingit possible to achieve switching operation methods more accurately andin quick response.

In a seventh aspect based on the first aspect, the motion informationobtaining means repeatedly obtains data indicating a position designatedby the pointing device, as the motion information. The movement amountcalculating means calculates the movement amount based on a changeamount of the designated positions of the pointing device.

According to the seventh aspect, the change amount of the designatedpositions of the pointing device is used to determine whether or not anoperation method employing the pointing device is to be used. Thereby,it is possible to more accurately sense a motion applied to the inputdevice housing itself.

In an eighth aspect based on the first aspect, the informationprocessing program causes the computer to further function as anaccumulation means (S49) for accumulating the movement amountscalculated by the movement amount calculating means. The thresholddetermining means determines whether or not the cumulative value of theaccumulation means exceeds a predetermined threshold.

According to the eighth aspect, a timing at which the user desires toswitch the operation methods can be more accurately determined.

In a ninth aspect based on the eighth aspect, the accumulation meansaccumulates the movement amounts For a predetermined period of timeuntil a current time.

According to the ninth aspect, when a small motion is continuallyapplied to the first input device housing for a long time, it ispossible to prevent from erroneously switching to the first positiondesignating means.

In a tenth aspect based on the eighth aspect, the information processingprogram causes the computer to further function as a cumulative valueresetting means (S28) for resetting the cumulative value when an inputhas occurred with respect to an input means other than the pointingdevice.

According to the tenth aspect, the operation methods can be moreaccurately switched.

In an eleventh aspect based on the first aspect, the informationprocessing program causes the computer to further function as a movementchange amount calculating means (S48) for calculating a change amount ofthe movement amounts calculated by the movement amount calculatingmeans. The threshold determining means determines whether or not thechange amount calculated by the movement change amount calculating meansexceeds a predetermined threshold.

In a twelfth aspect based on the eleventh aspect, the informationprocessing program causes the computer to further function as a changeamount accumulating means (S49) for accumulating the change amountscalculated by the movement change amount calculating means. Thethreshold determining means determines whether or not the cumulativechange amount of the change amount accumulating means exceeds apredetermined threshold.

According to the eleventh and twelfth aspects, a motion applied to aninput device housing can be detected based on a change amount ofmovements, thereby making is possible to more accurately detect amovement amount.

In a thirteenth aspect of the twelfth aspect, the change amountaccumulating means accumulates the change amounts for a predeterminedperiod of time until a current time.

According to the thirteenth aspect, when a small motion is continuallyapplied to the first input device housing for a long time, it ispossible to prevent from erroneously switching to the first positiondesignating means.

In a fourteenth aspect based on the twelfth aspect, the informationprocessing program causes the computer to further function as acumulative value resetting means (S28) for resetting the cumulativechange amount when an input has occurred with respect to an input meansother than the pointing device.

According to the fourteenth aspect, the operation methods can be moreaccurately switched.

In a fifteenth aspect based on the first aspect, the first input devicehousing includes an image capturing means for capturing an image of atleast one object whose image to be captured and which is located in avicinity of a display device. The first position designating meansincludes a captured image data obtaining means (S21). The captured imagedata obtaining means obtains captured image data output from the firstinput device housing including the image capturing means. The firstposition designating means designates a position based on an objectwhose image to be captured is shown in a captured image indicated by thecaptured image data.

According to the fifteenth aspect, when a pointing device is used whichis operated while an input device is directed toward a display device,the operation methods can be switched with further improved operabilityand in quick response.

A sixteenth aspect of the present invention is directed to acomputer-readable recording medium recording an information processingprogram executable by a computer of an information processing apparatusfor selecting one of a plurality of objects displayed on a screen, theprogram causing the computer to function as an motion informationobtaining means (S47) and a selection mode switching means (S23, S27,S52 to S54). The motion information obtaining means obtains motioninformation that is information about a motion applied to apredetermined input device housing itself used for an operation ofselecting the objects. The selection mode switching means switches aselection mode based on the motion information, between a pointingselection mode and a key selection mode, the pointing selection modebeing a selection mode in which any position on the screen can bedesignated, and any of the objects displayed on the screen can beselected by designating a position where the object is displayed, andthe key selection mode being a selection mode in which setting of afocused state indicating a selected state is changed among the objectsdisplayed on the screen in a predetermined order, depending on apredetermined key input.

According to the sixteenth aspect, a method employing a pointingtechnique and a method without it are switched based on a motion appliedto an input device housing itself, thereby making it possible to achievea switching operation with quiker response.

In a seventeenth aspect based on the sixteenth aspect, the informationprocessing program causes the computer to further function as a firstdetermination means (S23) and a movement amount calculating means (S48).The first determination means determines whether or not thepredetermined key input has been performed when the selection mode isthe pointing selection mode. The movement amount calculating meanscalculates a movement amount of the input device itself based on themotion information when the selection mode is the key selection mode.The selection mode switching means switches the selection mode to thekey selection mode when the first determination means determines thatthe key input has been performed, and switches the selection mode to thepointing selection mode when the movement amount calculated by themovement amount calculating means exceeds a predetermined threshold.

According to the seventeenth aspect, the selection methods can beswitched using an operation natural to the user.

In an eighteenth aspect based on the sixteenth aspect, the input devicehousing includes a motion sensor for detecting a motion applied to theinput device housing itself. The motion information obtaining meansobtains data output from the motion sensor as the motion information.

According to the eighteenth aspect, a motion applied to an input deviceitself can be more accurately sensed.

In a nineteenth aspect based on the sixteenth aspect, the informationprocessing program causes the computer to further function as anaccumulation means (S49) for accumulating the movement amountscalculated by the movement change amount calculating means. Theselection mode switching means switches the selection mode to thepointing selection mode when the cumulative value of the accumulationmeans exceeds a predetermined threshold.

According to the nineteenth aspect, a timing at which the user desiresto switch the operation methods can be more accurately determined.

In a twentieth aspect based on the nineteenth aspect, the accumulationmeans accumulates the movement amounts for a predetermined period oftime until a current time.

According to the twentieth aspect, when a small motion is continuallyapplied to an input device housing for a long time, it is possible toprevent from erroneously switching the selection modes.

In a twenty-first aspect based on the nineteenth aspect, the informationprocessing program causes the computer to further function as acumulative value resetting means (S28) for resetting the cumulativevalue when the first determination means determines that thepredetermined key input has been performed.

According to the twenty-first aspect, the operation methods can be moreaccurately switched.

In a twenty-second aspect based on the sixteenth aspect, the informationprocessing program causes the computer to further function as a movementchange amount calculating means for calculating a change amount of themovement amounts calculated by the movement amount calculating means.The selection mode switching means switches the selection mode to thepointing selection mode when the change amount calculated by themovement change amount calculating means exceeds a predeterminedthreshold.

In a twenty-third aspect based on the twenty-second aspect, theinformation processing program causes the computer to further functionas a change amount accumulating means for accumulating the changeamounts calculated by the movement change amount calculating means. Theselection mode switching means switches the selection mode to thepointing selection mode when the cumulative change amount of the changeamount accumulating means exceeds a predetermined threshold.

According to the twenty-second and twenty-third aspects, a motionapplied to an input device housing can be detected based on a movementchange amount, thereby making it possible to more accurately determine amovement amount.

In a twenty-fourth aspect based on the twenty-third aspect, the changeamount accumulating means accumulates the change amounts for apredetermined period of time until a current time.

According to the twenty-fourth aspect, when a small motion iscontinually applied to an input device housing for a long time, it ispossible to prevent from erroneously switching the selection modes.

In a twenty-fifth aspect based on the twenty-third aspect, theinformation processing program causes the computer to further functionas a cumulative value resetting means for resetting the cumulativechange amount when an input has occurred with respect to an input meansother than the pointing device.

According to the twenty-fifth aspect, the operation methods can be moreaccurately switched.

In a twenty-sixth aspect based on the seventeenth aspect, theinformation processing program causes the computer to further functionas a no-input period measuring means (S46, S53) for measuring a periodof time during which it is determined that the key input is not beingperformed when the selection mode is the key selection mode. Theselection mode switching means switches the selection mode to thepointing selection mode when the period of time measured by the no-inputperiod measuring means exceeds a predetermined value.

According to the twenty-sixth aspect, the convenience for the user canbe further improved.

A twenty-seventh aspect of the present invention is directed to aninformation processing apparatus for designating a position on a screenbased on outputs of a plurality of input means including at least onepointing device, including an motion information obtaining means (10), amovement amount calculating means (10), a threshold determining means(10), and a first position designating means (10). The motioninformation obtaining means obtains motion information that isinformation about a motion applied to a first input device housingitself including the pointing device. The movement amount calculatingmeans calculates a movement amount of the first input device housingbased on the motion information. The threshold determining meansdetermines whether or not the movement amount exceeds a predeterminedthreshold. The first position designating means designates a positionbased on an output from the pointing device when the thresholddetermining means determines that the movement amount exceeds thepredetermined threshold.

According to the twenty-seventh aspect, an effect similar to that of thefirst aspect is obtained.

A twenty-eighth aspect of the present invention is directed to aninformation processing apparatus for selecting one of a plurality ofobjects displayed on a screen, including an motion information obtainingmeans (10) and a selection mode switching means (10). The motioninformation obtaining means obtains motion information that isinformation about a motion applied to a predetermined input devicehousing itself used for an operation of selecting the objects. Theselection mode switching means switches a selection mode based on themotion information, between a pointing selection mode and a keyselection mode, the pointing selection mode being a selection mode inwhich any position on the screen can be designated, and any of theobjects displayed on the screen can be selected by designating aposition where the object is displayed, and the key selection mode beinga selection mode in which setting of a focused state indicating aselected state is changed among the objects displayed on the screen in apredetermined order, depending on a predetermined key input.

According to the twenty-eighth aspect, an effect similar to that of thesixteenth aspect can be obtained.

According to the present invention, a plurality of position designatingmethods employing a pointing device can be switched in quick response,thereby making it possible to improve operability and convenience forthe player.

These and other objects, features, aspects and advantages of the presentinvention will become more apparent from the following detaileddescription of the present invention when taken in conjunction with theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an external view of a game system 1 according to an embodimentof the present invention;

FIG. 2 is a functional block diagram of a game apparatus main body 3;

FIG. 3 is a perspective view showing a controller 7 of FIG. 1 as viewedfrom the top and the rear;

FIG. 4 is a perspective view showing the controller 7 of FIG. 3 asviewed from the bottom and the front;

FIG. 5 is a perspective view of the controller 7 of FIG. 3 where anupper housing thereof is cut away;

FIG. 6 is a perspective view of the controller 7 of FIG. 3 where a lowerhousing thereof is cut away;

FIG. 7 is a block diagram showing a configuration of the controller 7 ofFIG. 3;

FIG. 8 is a diagram showing an exemplary captured image;

FIG. 9 is a diagram for describing a game operation using the controller7;

FIG. 10 is a diagram showing an exemplary game screen assumed in thisembodiment;

FIG. 11 is a diagram showing an exemplary game screen assumed in thisembodiment;

FIG. 12 is a diagram showing an exemplary game screen assumed in thisembodiment;

FIG. 13 is a diagram showing an exemplary game screen assumed in thisembodiment;

FIG. 14 is a diagram showing an exemplary game screen assumed in thisembodiment;

FIG. 15 is a diagram showing an exemplary game screen assumed in thisembodiment;

FIG. 16 is a diagram showing a memory map of a main memory 12;

FIG. 17 is a flowchart showing a game process according to an embodimentof the present invention;

FIG. 18 is a flowchart showing an initialization process shown in stepS1 of FIG. 17 in detail;

FIG. 19 is a flowchart showing a one-frame process shown in step S2 ofFIG. 17 in detail;

FIG. 20 is a flowchart showing a cross key mode process shown in stepS31 of FIG. 19 in detail;

FIG. 21 is a diagram showing the controller 7 to which an extensioncontroller 9 is connected; and

FIG. 22 is a diagram showing a manner of holding the controller 7 towhich an extension controller 9 is connected.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter, embodiments of the present invention will be described withreference to the accompanying drawings. Note that the present inventionis not limited to these examples.

(Whole Configuration of Game System)

A game system 1 including a game apparatus according to an embodiment ofthe present invention will be described with reference to FIG. 1. FIG. 1is an external view of the game system 1. Hereinafter, the gameapparatus of this embodiment and a game program will be described, wherethe game apparatus is a stationary game apparatus as an example. In FIG.1, the game system 1 includes a television set (hereinafter simplyreferred to as a “television”) 2, the game apparatus main body 3, anoptical disc 4, a controller 7, and a marker unit 8. In the game system1, the game apparatus main body 3 executes a game process based on agame operation using the controller 7.

The optical disc 4, which is an exemplary information storing mediumchangeable with respect to the game apparatus main body 3, is detachablyloaded into the game apparatus main body 3. A game program that isexecuted in the game apparatus main body 3 is stored on the optical disc4. On a front surface of the game apparatus main body 3, a slot throughwhich the optical disc 4 is inserted is provided. The game apparatusmain body 3 executes a game process by reading and executing the gameprogram stored on the optical disc 4 which has been inserted into theslot.

The television 2 as an exemplary display device is connected via aconnection cord to the game apparatus main body 3. The television 2displays a game image that is obtained as a result of the game processexecuted in the game apparatus main body 3. The maker unit 8 is providedin the vicinity of the screen of the television 2 (on an upper side ofthe screen in FIG. 1). The maker unit 8 comprises two markers 8R and 8Lat both ends thereof. Specifically, the marker 8R is one or moreinfrared LEDs that output infrared light toward the front of thetelevision 2 (the same is true of the marker 8L). The maker unit 8 isconnected to the game apparatus main body 3, so that the game apparatusmain body 3 can control ON/OFF of each infrared LED included in themaker unit 8.

The controller 7 is an input device which inputs operation dataindicating an operation performed with respect to the controller 7, tothe game apparatus main body 3. The controller 7 and the game apparatusmain body 3 are connected via wireless communication. In thisembodiment, for example, the Bluetooth(R) technology is used forwireless communication between the controller 7 and the game apparatusmain body 3. Note that, in another embodiment, the controller 7 and thegame apparatus main body 3 may be connected via wired communication.

(Internal Configuration of Game Apparatus Main Body 3)

Next, an internal configuration of the game apparatus main body 3 willbe described with reference to FIG. 2. FIG. 2 is a block diagram showingthe configuration of the game apparatus main body 3. The game apparatusmain body 3 has a CPU 10, a system LSI 11, an external main memory 12, aROM/RTC 13, a disc drive 14, an AV-IC 15, and the like.

The CPU 10 executes a game program stored on the optical disc 4 toperform a game process, i.e., functions as a game processor. The CPU 10is connected to the system LSI 11. In addition to the CPU 10, theexternal main memory 12, the ROM/RTC 13, the disc drive 14, and theAV-IC 15 are connected to the system LSI 11. The system LSI 11 performsprocesses, such as controlling data transfer between each part connectedthereto, generating an image to be displayed, obtaining data from anexternal apparatus, and the like. An internal configuration of thesystem LSI 11 will be described below. The volatile external main memory12 stores a program, such as a game program read out from the opticaldisc 4, a game program read out from a flash memory 17, or the like, orvarious kinds of data, and is used as a work area or a buffer area forthe CPU 10. The ROM/RTC 13 has a ROM (so-called boot ROM) which stores aprogram for booting the game apparatus main body 3, and a clock circuit(RTC: Real Time Clock) which counts time. The disc drive 14 reads outprogram data, texture data or the like from the optical disc 4, andwrites the read data into an internal main memory 11 e (described below)or the external main memory 12.

The system LSI 11 also includes an input/output processor 11 a, a GPU(Graphics Processor Unit) 11 b, a DSP (Digital Signal Processor) 11 c, aVRAM 11 d, and the internal main memory 11 e. The parts 11 a to 11 e areconnected with each other via an internal bus (not shown).

The GPU 11 b, which is a part of a drawing means, generates an image inaccordance with a graphics command (image drawing command) from the CPU10. More specifically, the GPU 11 b performs a calculation processrequired to display 3D graphics, such as coordinate conversion from 3Dcoordinates to 2D coordinates (preprocess before rendering) or the like,and a final rendering process, such as attaching texture or the like, inaccordance with the graphics command, to generate game image data. Here,the CPU 10 inputs, to the GPU 11 b, an image generating program requiredto generate game image data in addition to the graphics command. TheVRAM 11 d stores data (e.g., polygon data, texture data, etc.) which isrequired by the GPU 11 b to execute the graphics command. When an imageis generated, the GPU 11 b generates image data using data stored in theVRAM 11 d.

The DSP 11 c, which functions as an audio processor, generates audiodata using sound data or sound waveform (tone color) data stored in theinternal main memory 11 e or the external main memory 12. The internalmain memory 11 e also stores a program or various data as with theexternal main memory 12, i.e., is also used as a work area or a bufferarea for the CPU 10.

The image data and audio data thus generated are read out by the AV-IC15. The AV-IC 15 outputs the read image data via an AV connector 16 tothe television 2, and the read audio data to a loudspeaker 2 a includedin the television 2. Thereby, an image is displayed on the television 2while a sound is output from the loudspeaker 2 a.

The input/output processor (I/O processor) 11 a executes datatransmission and reception between parts connected thereto, or downloadsdata from an external apparatus. The input/output processor 11 a isconnected to the flash memory 17, a wireless communications module 18, awireless controller module 19, an extension connector 20, and anexternal memory card connector 21. An antenna 22 is connected to thewireless communications module 18, and an antenna 23 is connected to thewireless controller module 19.

The input/output processor 11 a is connected via the wirelesscommunications module 18 and the antenna 22 to a network, and cancommunicate with other game apparatuses or various servers connected tothe network. The input/output processor 11 a regularly accesses theflash memory 17 to detect the presence or absence of data that needs tobe transmitted to the network. In the case of the presence of the data,the input/output processor 11 a transmits the data via the wirelesscommunications module 18 and the antenna 22 to the network. Theinput/output processor 11 a also receives data transmitted from othergame apparatuses or data downloaded from a download server via thenetwork, the antenna 22, and the wireless communications module 18, andstores the received data into the flash memory 17. The CPU 10 executes agame program to read out the data stored in the flash memory 17 andutilizes the data in the game program. In addition to data communicatedbetween the game apparatus main body 3 and other game apparatuses orvarious servers, save data (result data or intermediate data of a game)of a game played using the game apparatus main body 3 may be stored intothe flash memory 17.

The input/output processor 11 a also receives, via the antenna 23 andthe wireless controller module 19, operation data transmitted from thecontroller 7, and stores (temporarily stores) the operation data into abuffer area of the internal main memory 11 e or the external main memory12.

Also, the extension connector 20 and the external memory card connector21 are connected to the input/output processor 11 a. The extensionconnector 20 is a connector for interface, such as USB or SCSI. When amedium (e.g., an external storage medium, etc.), a peripheral device(e.g., another controller, etc.), or a wired communications connector isconnected to the extension connector 20, communication with a networkcan be performed without using the wireless communications module 18.The external memory card connector 21 is a connector for connecting anexternal storage medium, such as a memory card or the like. For example,the input/output processor 11 a can access an external storage mediumvia the extension connector 20 or the external memory card connector 21to save data or read out data.

The game apparatus main body 3 is provided with a power button 24, areset button 25, and an eject button 26. The power button 24 and thereset button 25 are connected to the system LSI 11. When the powerbutton 24 is turned ON, power is supplied via an AC adaptor (not shown)to each part of the game apparatus main body 3. Also, if the powerbutton 24 is pressed again while the power supply is ON, the gameapparatus main body 3 is transitioned to a low-power standby mode. Evenin this state, the game apparatus main body 3 is energized, so that thegame apparatus main body 3 can always be connected to a network, such asthe Internet or the like. Note that the power supply which is currentlyON can be turned OFF by pressing the power button 24 for a predeterminedtime or more. When the reset button 25 is pressed, the system LSI 11reboots the boot program of the game apparatus main body 3. The ejectbutton 26 is connected to the disc drive 14. When the eject button 26 ispressed, the optical disc 4 is ejected from the disc drive 14.

Next, the controller 7 will be described with reference to FIGS. 3 and4. FIG. 3 is a perspective view of the controller 7 as viewed from thetop and the rear. FIG. 4 is a perspective view of the controller 7 asviewed from the bottom and the front.

In FIGS. 3 and 4, the controller 7 has a housing 71, and an operationsection 72 comprising a plurality of operation buttons provided on asurface of the housing 71. The housing 71 of this example is in theshape of substantially a rectangular parallelepiped where afront-to-rear direction thereof is a longitudinal direction thereof. Thewhole housing 71 has a size which allows an adult and a child to holdthe controller 7 with one hand. The housing 71 is formed by, forexample, plastic molding.

A cross key 72 a is provided on a central portion closer to a frontsurface of an upper surface of the housing 71. The cross key 72 a is afour-direction push switch in the shape of a cross, and has operationportions corresponding to the respective four directions (frontward,rearward, leftward, and rightward), the operation portions beingprovided at the respective projecting pieces of the cross which arearranged at intervals of 900. Any one of the frontward, rearward,leftward and rightward directions is selected by a player pushing down acorresponding one of the operation portions of the cross key 72 a. Forexample, by a player operating the cross key 72 a, a movement directionof a player character or the like appearing in a virtual game world canbe designated, or one can be selected and designated from a plurality ofoptions.

Note that the cross key 72 a is an operation section which outputs anoperation signal, depending on the aforementioned direction inputoperation by a player, or may be an operation section of otherembodiments. For example, an operation section may be provided in whichfour push switches are disposed in cross directions, and an operationsignal is output, depending on a push switch pushed down by the player.In addition to the four push switches, a center switch may be providedat an intersection position of the cross directions, i.e., a complexoperation section comprising the four push switches and the centerswitch may be provided. Also, instead of the cross key 72 a, anoperation section may be provided which outputs an operation signal,depending on a tilt direction of a tiltable stick (so-called joystick)which projects from the upper surface of the housing 71. Also, insteadof the cross key 72 a, an operation section may be provided whichoutputs an operation signal, depending on a slide direction of andisc-like member which can be moved in a horizontal direction. Also, atouch pad may be provided instead of the cross key 72 a.

A plurality of operation buttons 72 b to 72 g are provided at the rearof the cross key 72 a on the upper surface of the housing 71. Theoperation buttons 72 b to 72 g are operation sections which outputoperation signals assigned to the respective operation buttons 72 b to72 g when a player pushes down the head portions of the respectivebuttons. For example, functions of a 1st button, a 2nd button, an Abutton, and the like are assigned to the operation buttons 72 b to 72 d.Functions of a minus switch, a home switch, a plus button, and the likeare assigned to the operation buttons 72 e to 72 g. The operationbuttons 72 a to 72 g are assigned the respective functions, depending ona game program executed by the game apparatus main body 3. Note that, inthe exemplary arrangement of FIG. 3, the operation buttons 72 b to 72 dare aligned on a central portion in a front-to-rear direction of theupper surface of the housing 71. The operation buttons 72 e to 72 g arealigned in a lateral direction on the upper surface of the housing 71and between the operation buttons 72 b and 72 d. The operation button 72f is a button of a type whose upper surface is buried below the uppersurface of the housing 71 so that the player is prevented fromunintentionally and erroneously pushing down the button.

Also, an operation button 72 h is provided at the front of the cross key72 a on the upper surface of the housing 71. The operation button 72 his a power supply switch which remotely switches ON/OFF a power supplyfor the game apparatus main body 3. The operation button 72 h is also abutton of the type whose upper surface is buried below the upper surfaceof the housing 71 so that a player is prevented from unintentionally anderroneously pushing down the button.

A plurality of LEDs 702 are provided at the rear of the operation button72 c on the upper surface of the housing 71. Here, the controller 7 isassigned controller identification (number) so as to distinguish it fromother controllers 7. For example, the LEDs 702 are used so as to notifya player of controller identification currently set for the controller7. Specifically, when transmission data is transmitted from thecontroller 7 to the game apparatus main body 3, one of the LEDs 702 isturned ON, depending on the controller identification.

Also, sound holes through which sound is emitted from a loudspeaker (aloudspeaker 706 of FIG. 5) described below to the outside are formedbetween the operation button 72 b and the operation buttons 72 e to 72 gon the upper surface of the housing 71.

On the other hand, a hollow portion is formed on a lower surface of thehousing 71. The hollow portion on the lower surface of the housing 71 isformed at a position where the index finger or the middle finger of aplayer is placed when the player holds the controller 7 with one handwhile directing the front surface of the controller 7 toward the markers8L and 8R. An operation button 72 i is provided on a slope surface ofthe hollow portion. The operation button 72 i is an operation sectionwhich functions as, for example, a B button.

An image capturing device 743 which is a part of the image captureinformation computing section 74 is provided on a front surface of thehousing 71. Here, the image capture information computing section 74 isa system for analyzing image data captured by the controller 7 todetermine a place having a high luminance in the image data and detect acenter-of-gravity position, a size or the like of the place. The imagecapture information computing section 74 has, for example, a maximumsampling cycle of about 200 frames/sec, and therefore, can track andanalyze relatively high-speed movement of the controller 7. A detailedstructure of the image capture information computing section 74 will bedescribed below. A connector 73 Is provided on a rear surface of thehousing 71. The connector 73 is, for example, an edge connector which isutilized so as to engage and connect the controller 7 with a connectioncable.

Here, in order to specifically describe the present invention, acoordinate system which is set with respect to the controller 7 isdefined as follows. As illustrated in FIGS. 3 and 4, X, Y and Z axes,which are orthogonal to each other, are defined with respect to thecontroller 7. Specifically, a front-to-rear direction of the controller7 (the longitudinal direction of the housing 71) is assumed to be the Zaxis, and a Front surface (a surface on which the image captureinformation computing section 74 is provided) direction of thecontroller 7 is assumed to be the positive direction of the Z axis. Avertical direction of the controller 7 is assumed to be the Y axis, anda lower surface (a surface on which the operation button 72 i isprovided) direction of the housing 71 is assumed to be the positivedirection of the Y axis. A lateral direction of the controller 7 isassumed to be the X axis, and a left side surface (a side surfaceillustrated in FIG. 4, but not in FIG. 3) direction of the housing 71 isassumed to be the positive direction of the X axis.

Next, an internal structure of the controller 7 will be described withreference to FIGS. 5 and 6. Note that FIG. 5 is a perspective view ofthe controller 7 (as viewed from a rear surface side thereof) where anupper housing (a portion of the housing 71) is cut away. FIG. 6 is aperspective view of the controller 7 as viewed from the front surfaceside thereof where a lower housing (a portion of the housing 71) is cutaway. FIG. 6 illustrates a perspective view of a base board 700 of FIG.5 as viewed from a bottom surface thereof.

In FIG. 5, the base board 700 is fixed inside the housing 71. On anupper major surface of the base board 700, the operation buttons 72 a to72 h, an acceleration sensor 701, the LEDs 702, an antenna 754, and thelike are provided. These are connected to a microcomputer 751 (see FIGS.6 and 7) via conductors (not shown) formed on the base board 700 and thelike. The microcomputer 751 functions to generate operation button data,depending on the operation button 72 a or the like, as an exemplarybutton data generating means of the present invention. This mechanism,which is a known technique, is implemented, for example, by themicrocomputer 751 detecting contact/disconnection of a conductor by aswitching mechanism, such as a tactile switch or the like, which isprovided under a key top. More specifically, for example, when anoperation button is pushed down to contact a conductor, a current flows.The microcomputer 751 detects the current flow to determine whichoperation button has been pushed down, and generates a signal, dependingon the operation button.

Also, by a radio module 753 (see FIG. 7) and an antenna 754, thecontroller 7 functions as a wireless controller. Note that a quartzoscillator (not shown) is provided in the housing 71, and generates abasic clock for the microcomputer 751 (described below). Also, theloudspeaker 706 and an amplifier 708 are provided on the upper majorsurface of the base board 700. Also, the acceleration sensor 701 isprovided on a left side of the operation button 72 d on the base board700 (i.e., a peripheral portion of the base board 700, but not a centerportion thereof) Therefore, the acceleration sensor 701 can detect anacceleration including a component due to a centrifugal force as well asa change in direction of a gravitational acceleration, depending on arotation of the controller 7 around the longitudinal direction as anaxis. Therefore, by predetermined computation, the game apparatus mainbody 3 or the like can determine the rotation of the controller 7 basedon the detected acceleration data with satisfactory sensitivity.

On the other hand, in FIG. 6, the image capture information computingsection 74 is provided at a front edge on a lower major surface of thebase board 700. The image capture information computing section 74comprises an infrared filter 741, a lens 742, the image capturing device743, and an image processing circuit 744, which are arranged in thisorder from the front of the controller 7, and are attached to the lowermajor surface of the base board 700. The connector 73 is attached to arear edge on the lower major surface of the base board 700. Also, asound IC 707 and the microcomputer 751 are provided on the lower majorsurface of the base board 700. The sound IC 707 is connected to themicrocomputer 751 and the amplifier 708 via conductors formed on thebase board 700 and the like, and outputs an audio signal via theamplifier 708 to the loudspeaker 706, depending on sound datatransmitted from the game apparatus main body 3.

A vibrator 704 is attached onto the lower major surface of the baseboard 700. The vibrator 704 may be, for example, a vibration motor or asolenoid. The vibrator 704 is connected to the microcomputer 751 via aconductor formed on the base board 700 and the like, and its activationis switched ON/OFF, depending on vibration data transmitted from thegame apparatus main body 3. The activation of the vibrator 704 generatesvibration in the controller 7, so that the vibration is transferred to aplayer's hand holding the controller 7, thereby making it possible toachieve a so-called vibration-feature supporting game. Here, since thevibrator 704 is disposed somehow closer to the front of the housing 71,the housing 71 significantly vibrates while the player is holding thecontroller 7, so that the player easily feels vibration.

Next, an Internal configuration of the controller 7 will be describedwith reference to FIG. 7. Note that FIG. 7 is a block diagramillustrating the configuration of the controller 7.

In FIG. 7, the controller 7 comprises a communication section 75 inaddition to the operation section 72, the image capture informationcomputing section 74, the acceleration sensor 701, the vibrator 704, theloudspeaker 706, the sound IC 707, and the amplifier 708.

The image capture information computing section 74 includes the infraredfilter 741, the lens 742, the image capturing device 743, and the imageprocessing circuit 744. The infrared filter 741 passes only infraredlight entering from the front of the controller 7. Here, the markers 8Land 8R which are provided in the vicinity of the display screen of thetelevision 2 are infrared LEDs which emit infrared light toward thefront of the television 2. Therefore, by providing the infrared filter741, images of the markers 8L and 8R can be accurately captured. Thelens 742 collects infrared light passing through the infrared filter 741and causes the light to enter the image capturing device 743. The imagecapturing device 743 may be, for example, a solid-state image capturingdevice, such as a CMOS sensor or a CCD, and captures the infrared lightcollected by the lens 742. Therefore, the image capturing device 743captures only infrared light passing through the infrared filter 741 togenerate image data. An image captured by the image capturing device 743is hereinafter referred to as a captured image. The image data generatedby the image capturing device 743 is processed by the image processingcircuit 744. The image processing circuit 744 calculates a position ofan object whose image to be captured (the markers 8L and 8R) in thecaptured image. Hereinafter, a method of calculating the position of anobject whose image to be captured will be described with reference toFIG. 8.

FIG. 8 is a diagram showing an exemplary captured image. In the capturedimage of FIG. 8, an image 8L′ of the marker 8L and an image 8R′ of themarker 8R are laterally arranged. When receiving the captured image, theimage processing circuit 744 initially calculates the coordinates of aposition of each region in the captured image that matches predeterminedconditions. Here, the predetermined conditions are conditions forfinding an image of an object to be captured (a target image).Specifically, the predetermined conditions are such that the region hasa luminance higher than or equal to a predetermined value (a highluminance region) and a size within a predetermined range. Note that anypredetermined conditions that can be used to Find an object whose imageto be captured may be employed. In other embodiments, the predeterminedconditions may relate to the color of an image.

When a position of a target image is calculated, the image processingcircuit 744 initially finds the aforementioned high luminance regions ascandidates for the target image, from the entire region of a capturedimage. This is because a target image appears as a high illuminanceregion in the image data of a captured image. Next, based on a size ofthe high luminance region thus found, the image processing circuit 744determines whether or not the high luminance region is the target image.The captured image may contain an image caused by sunlight through awindow or light of a fluorescent tube in a room in addition to theimages (target images) 8L′ and 8R′ of the two markers 8L and 8R. In thiscase, such an image may appear as a high illuminance region in additionto the images of the markers 8L and 8R. The aforementioned determinationprocess is for distinguishing the images of the markers 8L and 8R(target images) from other images to accurately find the target images.Specifically, in the determination process, it is determined whether ornot a high luminance region thus found has a size within a predeterminedrange. When the high luminance region has a size within thepredetermined range, the high luminance region is determined to be thetarget image. When the size of the high luminance region is not withinthe predetermined range, the high luminance region is determined to bean image other than the target image.

Further, for a high luminance region which is determined to representthe target image as a result of the determination process, the imageprocessing circuit 744 calculates a position of the high luminanceregion. Specifically, a position of the center of gravity of the highluminance region is calculated. Note that the position of the center ofgravity can be calculated with a scale finer than the resolution of theimage capturing element 743. It is here assumed that an image capturedby the image capturing element 743 has a resolution of 126×96, and theposition of the center of gravity can be calculated with a scale of1024×768. In this case, the coordinates of the position of the center ofgravity is represented with integer values in the range of (0, 0) to(1024, 768). Note that the position of the captured image is assumed tobe represented by a coordinate system (xy coordinate system) where theupper left corner of the captured image is the origin, the downwarddirection is the positive direction of the y axis, and the rightwarddirection is the positive direction of the x axis.

As described above, the image processing circuit 744 calculatescoordinates indicating a position of each region which satisfies thepredetermined conditions in the captured image. Note that thecoordinates calculated by the image processing circuit 744 are referredto as marker coordinates. The marker coordinates are coordinates thatindicate a position of an object whose image to be captured in thecoordinate system for representing a position on a plane correspondingto the captured image. The image processing circuit 744 outputs themarker coordinates to the microcomputer 751 of the communication section75. The data of the marker coordinates is transmitted as operation datato the game apparatus main body 3 by the microcomputer 751. Since themarker coordinates vary depending on an orientation (attitude) or aposition of the controller 7 itself, the game apparatus main body 3 cancalculate the orientation or the position of the controller 7. Althoughthe processes until marker coordinates are calculated from a capturedimage are carried out by the image processing circuit 744 and/or themicrocomputer 751 of the controller 7 in this embodiment, the capturedimage may be transferred to the game apparatus main body 3 and processessimilar to the subsequent processes may be executed by the CPU 10 or thelike of the game apparatus main body 3.

The controller 7 preferably comprises the acceleration sensor 701 whichsenses accelerations along with three axes (x, y and z axes). Thethree-axis acceleration sensor 701 senses linear accelerations in threedirections, i.e., a vertical direction, a lateral direction, and afront-to-rear direction. In another embodiment, the acceleration sensor701 may be a two-axis acceleration detecting means which senses onlylinear accelerations along two axes in the vertical direction and thelateral direction (or other direction pairs), depending on the type of acontrol signal used in a game process. For example, the three- ortwo-axis acceleration sensors 701 may be of a type which is availablefrom Analog Devices, Inc. or STMicroelectronics N.V. The accelerationsensor 701 may be of a capacitance type (capacitance coupling type)based on a technique of MEMS (Micro Electro Mechanical Systems) obtainedby micromachining a silicon material However, the three- or two-axisacceleration sensor 701 may be implemented using a technique of anexisting acceleration detecting means (e.g., a piezoelectric type or apiezoelectric resistance type) or other appropriate techniques whichwill be developed in the future.

It is known to those skilled in the art that an acceleration detectingmeans as used in the acceleration sensor 701 can sense only anacceleration (linear acceleration) along a straight line correspondingto each axis of the acceleration sensor 701. In other words, a directoutput from the acceleration sensor 701 is a signal indicating a linearacceleration (static or dynamic) along each of the two or three axes.Therefore, the acceleration sensor 701 cannot directly sense physicalcharacteristics, such as a motion along a non-linear path (e.g., an arc,etc.), a rotation, a rotational motion, an angular displacement, a tilt,a position, an attitude, and the like.

However, it can be easily understood by those skilled in the art fromthe description of the present specification that, by a computer, suchas a processor (e.g., the CPU 10) of the game apparatus, a processor(e.g., the microcomputer 751) of the controller 7 or the like, executinga process based on a signal relating to an acceleration output from theacceleration sensor 701, additional information about the controller 7can be estimated or calculated (determined). For example, a process maybe performed by the computer, assuming that the controller 7 having theacceleration sensor 701 is in the static state (i.e., assuming that anacceleration detected by the acceleration sensor 701 is only agravitational acceleration). In this case, if the controller 7 isactually in the static state, it can be determined based on the detectedacceleration whether or not or how much the attitude of the controller 7is tilted with respect to the gravity direction. Specifically, if astate of the acceleration sensor 701 whose detected axis is orientedvertically downward is assumed as a reference, it can be determinedwhether or not the controller 7 is tilted, based on whether or not 1 G(gravitational acceleration) is applied to the acceleration sensor 701,and it can be determined how much the controller is tilted, based on themagnitude of the acceleration detected by the acceleration sensor 701.Also, in the case of a multi-axis acceleration sensor, by subjecting anacceleration signal of each axis to a process, it can be determined inmore detail how much the controller 7 is tilted with respect to thegravity direction. In this case, a processor may perform a process ofcalculating data about a tilt angle of the controller 7 based on theoutput of the acceleration sensor 701. Alternatively, a process ofapproximately estimating the tilt may be performed based on the outputof the acceleration sensor 701 without performing the process ofcalculating the data about the tilt angle. Thus, by using a processor incombination with the acceleration sensor 701, a tilt, an attitude or aposition of the controller 7 can be determined. On the other hand, whenit is assumed that the acceleration sensor 701 is in the dynamic state,an acceleration depending on a motion of the acceleration sensor 701 isdetected in addition to the gravitational acceleration component.Therefore, a motion direction or the like can be determined by removingthe gravitational acceleration component by a predetermined process.Specifically, when the controller 7 comprising the acceleration sensor701 is dynamically accelerated by a user's hand, various motions and/orpositions of the controller 7 can be calculated by processing anacceleration signal generated by the acceleration sensor 701. Note that,even if it is assumed that the acceleration sensor 701 is in the dynamicstate, a tilt with respect to the gravity direction can be determined byremoving an acceleration depending on a motion of the accelerationsensor 701 by a predetermined process. In another example, theacceleration sensor 701 may comprise a built-in signal processing deviceor another type of dedicated processing device for performing a desiredprocess with respect to an acceleration signal output from a built-inacceleration detecting means before outputting a signal to themicrocomputer 751. For example, if the acceleration sensor 701 detects astatic acceleration (e.g., the gravitational acceleration), the built-inor dedicated processing device may convert a sensed acceleration signalinto a tilt angle corresponding thereto (or another preferableparameter).

In another embodiment, as an acceleration sensor for detecting a motionof the controller 7, a gyro-sensor comprising a rotation element, avibration element, or the like may be employed. An exemplary MEMSgyro-sensor used in this embodiment is available from Analog Devices,Inc. As is different from the acceleration sensor 701, the gyro-sensorcan directly sense a rotation (or an angular velocity) about an axis ofat least one gyro-element included therein. Thus, since the gyro-sensorand the acceleration sensor are basically different from each other, oneof them is selected, depending on the individual application, andprocesses performed for output signals from these devices need to bechanged as appropriate.

Specifically, when a tilt or an attitude is calculated using agyro-sensor instead of an acceleration sensor, a significant change isrequired. Specifically, when a gyro-sensor is used, the value of a tiltis initialized during the start of detection. Thereafter, angularacceleration data output from the gyro-sensor is integrated. Next, theamount of a change in tilt is calculated from the initialized tiltvalue. In this case, the calculated tilt has a value corresponding to anangle. On the other hand, when an acceleration sensor is used tocalculate a tilt, the tilt is calculated by comparing the value of acomponent about each axis of a gravitational acceleration with apredetermined reference. Therefore, the calculated tilt can berepresented by a vector, and an absolute direction can be detected bythe acceleration detecting means without initialization. Also, whereas avalue calculated as a tilt is an angle when a gyro-sensor is used, thevalue is a vector when an acceleration sensor is used. Therefore, when agyro-sensor is used instead of an acceleration sensor, the tilt dataneeds to be subjected to predetermined conversion, taking intoconsideration a difference between the two devices. The characteristicsof gyro-sensors as well as the basic difference between the accelerationdetecting means and the gyro-sensor are known to those skilled in theart, and will not be herein described in more detail. Whereasgyro-sensors have an advantage of directly sensing rotation,acceleration sensors generally have an advantage over the gyro-sensor interms of cost effectiveness when the acceleration sensor is applied to acontroller as used in this embodiment.

The communication section 75 comprises the microcomputer 751, the memory752, the radio module 753, and the antenna 754. The microcomputer 751controls the radio module 753 for wirelessly transmitting transmissiondata while using the memory 752 as a memory area during a process. Also,the microcomputer 751 controls operations of the sound IC 707 and thevibrator 704, depending on data from the game apparatus main body 3which is received by the radio module 753 via the antenna 754. The soundIC 707 processes sound data or the like transmitted from the gameapparatus main body 3 via the communication section 75. Also, themicrocomputer 751 activates the vibrator 704, depending on vibrationdata (e.g., a signal for switching ON/OFF the vibrator 704) or the liketransmitted from the game apparatus main body 3 via the communicationsection 75.

An operation signal (key data) from the operation section 72 provided inthe controller 7, acceleration signals (x-, y- and z-axis directionacceleration data; hereinafter simply referred to as acceleration data)from the acceleration sensor 701, and process result data from the imagecapture information computing section 74, are output to themicrocomputer 751. The microcomputer 751 temporarily stores the receiveddata (the key data, the acceleration data, and the process result data),as transmission data to be transmitted to the wireless controller module19, into the memory 752. Here, radio transmission from the communicationsection 75 to the wireless controller module 19 is performed inpredetermined cycles. Since a game is generally processed in units of1/60 sec, the cycle of the radio transmission needs to be shorter than1/60 sec. Specifically, the game processing unit is 16.7 ms ( 1/60 sec),and the transmission interval of the communication section 75 employingBluetooth(R) is 5 ms. When timing of transmission to the wirelesscontroller module 19 arrives, the microcomputer 751 outputs transmissiondata stored in the memory 752, as a series of pieces of operationalinformation, to the radio module 753. Thereafter, the radio module 753modulates the operational information using a carrier wave having apredetermined frequency and emits the resultant radio signal from theantenna 754, by means of, for example, the Bluetooth(R) technique.Specifically, the key data from the operation section 72 provided in thecontroller 7, the acceleration data from the acceleration sensor 701,and the process result data from the image capture information computingsection 74 are modulated into a radio signal by the radio module 753,and the radio signal is transmitted from the controller 7. Thereafter,the wireless controller module 19 of the game apparatus main body 3receives the radio signal, and the game apparatus main body 3demodulates or decodes the radio signal, thereby obtaining a series ofpieces of operational information (the key data, the acceleration data,and the process result data) Thereafter, the CPU 10 of the gameapparatus main body 3 performs a game process based on the obtainedoperational information and a game program. Note that, when thecommunication section 75 is configured using the Bluetooth(R) technique,the communication section 75 can also have a function of receivingtransmission data wirelessly transmitted from other devices.

Here, a game operation using the controller 7 will be described. Whenthe controller 7 is used to play a game in the game system 1, a playerholds the controller 7 with one hand. In this case, as shown in FIG. 9,the player holds the controller 7 while directing the front surface (aside through which light to be captured by the image capture informationcomputing section 74 enters) of the controller 7 toward the markers 8Land 8R. In this state, the player performs a game operation by changinga tilt of the controller 7, a position (designated position) on thescreen pointed or designated by the controller 7, or a distance betweenthe controller 7 and each of the markers 8L and 8R.

Next, an outline of a process executed by the thus-configured gamesystem 1 will be described. The present invention is generallyapplicable to any process of “selecting” an object or the like displayedon the screen. In this embodiment, as an example of such a process, agame in which a plurality of minigames are enjoyed is assumed. Ofprocesses involved in the game, a process and an operation that areperformed on a menu screen so as to select a minigame desired by theplayer will be described as an example.

FIG. 10 is a diagram showing an example of the menu screen. In FIG. 10,objects to be selected 101 to 103 and a pointer 104 are displayed. Eachobject represents a corresponding one of the minigames. The playerselects any one of these objects 101 to 103 and presses down apredetermined determination button (e.g., the operation button 72 b),thereby starting a play of the desired minigame. In principle, thepointer 104 is displayed at a position (designated position) on thescreen pointed by the controller 7.

Next, selection operation modes (hereinafter simply referred to asoperation modes) in this embodiment will be described. In thisembodiment, two operation modes, i.e., a “pointing mode” and a “crosskey mode”, are employed. The pointing mode is an operation mode in whichthe pointer 104 is moved on the screen so as to select one of theobjects. The cross key mode is an operation mode in which the cross key72 a is pressed down so as to select one of the objects.

An operation of performing selection in the pointing mode will bedescribed with reference to FIGS. 10 to 15. Initially, as shown in FIG.10, the pointer 104 is assumed to be in a state in which the pointer 104is displayed in an upper left portion of the screen (a portion pointedby the controller 7). When from this state the pointer 104 (designatedposition) is moved to a position where the object 101 is displayed, aselection frame 105 is displayed which surrounds the object 101, and theobject 101 is selected, as shown in FIG. 11. When from this state thepointer 104 is moved to a position where no object is displayed, theselected state is canceled (the selection frame 105 is not displayed).When from the state of FIG. 11 the pointer 104 is moved to a positionwhere the object 102 is displayed, the selection frame 105 is displayedwhich surrounds the object 102, and the object 102 is selected, as shownin FIG. 12.

Next, an operation of performing selection in the cross key mode will bedescribed. For example, when the “right” portion of the cross key 72 ais pressed down by the player while the object 101 is selected as shownin FIG. 11, the object 102 displayed right-adjacent to the object 101 isthen selected as shown in FIG. 12. In other words, with reference to acurrently selected object, another object which is located in adirection of the cross key 72 a that is pressed will be selected.Moreover, in this case, the displayed position of the pointer 104 ischanged to a position that is located slightly left to and below amiddle of the selected object 102.

When the “left” portion of the cross key 72 a is pressed in the state ofFIG. 11, then the rightmost object 103 is selected, and the pointer 104is displayed at a position that is located slightly left to and below amiddle of the selected object 103, as shown in FIG. 14. In other words,in this embodiment, when the “right” portion is pressed while therightmost object 103 is selected, the leftmost object 101 is thenselected, and when the “left” portion is pressed while the leftmostobject 101 is selected, the rightmost object 103 is then selected.

Although not shown, when objects are vertically arranged, the objectscan be similarly selected along the vertical direction by pressing the“top” and “bottom” portions of the cross key 72 a.

Note that the aforementioned control of the order of selection in thecross key mode is only for illustrative purposes. For example, when the“right” portion is pressed while the rightmost object 103 is selected,the selected state may not be changed (i.e., the object 103 may remainselected).

Next, a method of switching the two operation modes will be described.In this embodiment, when the cross key 72 a is pressed in the pointingmode, the mode is switched to the cross key mode. Specifically, when theplayer presses the cross key 72 a in the state of FIG. 10, any of theobjects is selected (in this embodiment, the immediately previouslyselected object is assumed to be still selected). Here, when the crosskey mode is continued, the pointer 104 is not moved even if thedesignated position on the screen is changed. For example, even when thedesignated position of the controller 7 is moved upward on the screenwhile the object 102 is selected as shown in FIG. 13, the displayedposition of the pointer 104 is not moved and the object 102 remainsselected.

On the other hand, when the cross key mode is switched to the pointingmode, an operation of “swinging” the controller 7 is performed. In thisembodiment, based on acceleration data output from the accelerationsensor 701, it is detected whether or not the “swing” operation has beenperformed. Specifically, if an acceleration having a predeterminedmagnitude or more (generated by “swinging”) is detected within apredetermined time in the cross key mode, the operation mode is switchedto the pointing mode. For example, when the controller 7 is swung whilethe object 103 is selected as shown in FIG. 14, the operation mode isswitched to the pointing mode, and at the same time, the pointer 104 isdisplayed at a designated position detected at that time, as shown inFIG. 15.

Note that the cross key mode may be switched to the pointing mode whenthe cumulative value of movement amounts of the controller 7 that arecalculated based on the acceleration data exceeds a predeterminedthreshold. Alternatively, the cross key mode may be switched to thepointing mode when the amount of a change in acceleration per unit time(e.g., one frame) exceeds a predetermined threshold. Alternatively, thecross key mode may be switched to the pointing mode when the cumulativevalue of the change amounts exceeds a predetermined threshold.

Thus, by switching the two operation modes by “swinging” the controller7, the modes can be switched in quick response without an unnaturaloperation. Specifically, in this embodiment, since the controller 7 isheld and operated as shown FIG. 9, the swing operation (e.g., a snapoperation in which a wrist is twisted) is similar to a manner in whichthe controller 7 is moved by a pointing operation in that the controller7 is moved by using an elbow or a wrist while directing the tip of thecontroller 7 toward the screen. Therefore, there is a high level ofaffinity between the “swing” operation and the “pointing” operation.Therefore, the cross key mode can be switched to the pointing mode witha natural operation and quickly (after the swing operation, the pointingoperation can be performed without changing the holding manner of thecontroller 7 or the like). Moreover, the pointing mode can be switchedto the cross key mode only by pressing the cross key 72 a. Therefore,the pointing mode and the cross key mode can be switched in quickresponse.

Note that, in this embodiment, when the cross key 72 a is not operatedfor a predetermined or more in the cross key mode and the screen isbeing pointed by the controller 7, the cross key mode is alsoautomatically switched to the pointing mode, thereby enhancing theconvenience for the player.

Next, a game process executed by the game apparatus main body 3 will bedescribed in detail. Firstly, data that is to be stored in the externalmain memory 12 in the game process will be described. FIG. 16 is adiagram showing a memory map of the external main memory 12 of the gameapparatus main body 3. In FIG. 16, the external main memory 12 includesa program storage area 120 and a data storage area 124. The data storedin the program storage area 120 and the data storage area 124 is storedon the optical disc 4 and is transferred to the external main memory 12when a game program is executed.

A game program that is executed by the CPU 10 is stored in the programstorage area 120. The game program includes a pointer control program121, a one-frame process program 122, a cross key mode program 123, andthe like.

In the data storage area 124, data, such as operation data 125, aprevious acceleration vector 126, an acceleration array 127, anoperation mode 128, a no-input time 129 and the like, and in addition,various variables and flags that are used during execution of a programare stored.

The operation data 125 is operation data that is transmitted from thecontroller 7 to the game apparatus main body 3. As described above,operation data is transmitted once per 1/200 sec from the controller 7to the game apparatus main body 3, so that the operation data 125 storedin the main memory 12 is updated at this frequency. In this embodiment,only latest (last obtained) operation data may be stored in the mainmemory 12.

The operation data 125 includes acceleration data 1251, markercoordinate data 1252, and operation button data 1253. The accelerationdata 1251 is data that indicates an acceleration (acceleration vector)detected by the acceleration sensor 701. Here, the acceleration data1251 indicates a three-dimensional acceleration vector V1 that has, asits components, accelerations along the directions of the three axes (x,y, z) of FIG. 2. Also, in this embodiment, the magnitude of theacceleration vector VI detected by the acceleration sensor 701 when thecontroller 7 remains at rest is assumed to be “1”. In other words, themagnitude of the gravity acceleration detected by the accelerationsensor 701 is “1”.

The marker coordinate data 1252 is coordinates that are calculated bythe image processing circuit 744 of the image capture informationcomputing section 74, i.e., data indicating the marker coordinates. Themarker coordinates are represented by a two-dimensional coordinatesystem (the xy coordinate system of FIG. 8) for representing a positionon a plane corresponding to a captured image. Note that when images ofthe two markers 8R and 8L are captured by the image capturing device743, two marker coordinate points are calculated. On the other hand,when one of the markers 8R and 8L is not located within a range in whichan image can be captured by the image capturing device 743, the imagecapturing device 743 captures an image of only the other marker andcalculates only one marker coordinate point. When none of the markers 8Rand 8L is Located within the range in which an image can be captured bythe image capturing device 743, the image capturing device 743 capturesno marker image and calculates no marker coordinate point. Therefore,the marker coordinate data 1252 may indicate two marker coordinatepoints, only one marker coordinate point, or no marker coordinate point.

The operation button data 1253 is data that indicates input states ofthe operation buttons 72 a to 72 i. Therefore, the operation button data1253 also includes data that indicates a pressed state of the cross key72 a (which of the upward, downward, leftward and rightward directionsis pressed down).

The previous acceleration vector 126 is data that indicates anacceleration vector that is calculated immediately before theacceleration data 1251. More specifically, the unit time of a gameprocess of this embodiment is 1/60 sec, and processes described beloware also repeatedly performed in units of 1/60 sec. The accelerationdata is one that is calculated by a current process loop. The previousacceleration vector 126 is vector data that indicates an accelerationthat is calculated by the immediately previous process loop.

The acceleration array 127 is a set of data (one-dimensional array) thatis used in a process described below to determine whether to switch theoperation modes, and represents the history of change amounts of theacceleration. It may also be hereinafter assumed that the number ofelements in the array is N and is counted from “0”, and the array isrepresented by an acceleration array Q[0] to Q[N−1]. The storagetechnique is a so-called FIFO (First In First Out). The latest data isstored in Q[0] and the oldest data is stored in Q[N−1].

The operation mode 128 is a flag that indicates whether the currentoperation mode is in the “pointing mode” or in the “cross key mode”. Inthis embodiment, the operation mode 128 is set to be “0” in the“pointing mode”, and “1” in the “cross key”.

The no-input time 129 is a variable for counting a time for which thereis no input to the cross key 72 a in the cross key mode (a time forwhich the cross key 72 a is not pressed down).

Next, a game process that is executed by the game apparatus main body 3will be described with reference to FIGS. 17 to 20. When the gameapparatus main body 3 is powered ON, the CPU 10 of the game apparatusmain body 3 executes the boot program stored in the ROM/RTC 13, therebyinitializing units, such as the main memory 33 and the like. Thereafter,a game program stored on the optical disc 4 is read into the externalmain memory 12, and the CPU 10 starts execution of the game program.FIG. 17 is a flowchart showing a game process that is performed aftercompletion of the aforementioned process. A process loop of steps S2 andS3 of FIG. 17 is executed for each frame. Note that, in the flowchart ofFIG. 17, control and object selection of the pointer 104 that areperformed based on an operation of the controller 7 by executing apointer control program included in the game program, will be described,and other processes that are not directly related to the presentinvention will not be described in detail.

In FIG. 17, initially, the CPU 10 executes an initialization process(step S1). FIG. 18 is a flowchart showing a detail of the initializationprocess of step S1. In FIG. 18, initially, the CPU 10 initializes theacceleration array 127 (step S11). More specifically, the CPU 10 sets“0” into variables included in the acceleration array 127.

Next, the CPU 10 initializes the no-input time 129 (step S12).Specifically, the CPU 10 sets “0” into the no-input time 129.

Next, the CPU 10 executes a process of calculating a designatedcoordinates Pos (step S13). The designated coordinates Pos are avariable for representing a position that is pointed or designated bythe controller 7. The process of step S13 will be described in moredetail. Initially, the CPU 10 calculates the designated coordinates Posof the controller 7 from the operation data 125, based on the markercoordinate data 1252. Here, the designated coordinates Pos may becalculated by any calculation method. For example, the following methodis contemplated.

Hereinafter, an exemplary method of calculating the designatedcoordinates Pos will be described. The operation data 125 obtained fromthe controller 7 includes the marker coordinate data 1252 that is dataindicating the marker coordinates. The data indicates two markercoordinate points of the markers 8L and 8R (see FIG. 8) Initially, theCPU 10 calculates a middle point between the two marker coordinatepoints. The middle point is represented by the xy coordinate system forrepresenting a position on a plane corresponding to a captured image.Next, the CPU 10 converts coordinates indicating the position of themiddle point into coordinates in a coordinate system (x′y′ coordinatesystem) for representing a position on the screen of the television 2.This conversion can be carried out by using a function that converts thecoordinates of a middle point calculated from a captured image intocoordinates on the screen corresponding to an actual designated positionof the controller 7 when the captured image is taken. In this case, thedesignated position of the controller 7 is moved in a direction oppositeto the positions of the marker coordinate points in the captured image,i.e., the direction is vertically or laterally reversed. Thethus-calculated x′y′ coordinate values are the designated coordinatesPos of the controller 7. Note that, as described above, the markercoordinate data 1252 may indicate the absence of a marker coordinatepoint. In such a case, data (e.g., a NULL value) for indicating theabsence of a marker coordinate point is set into the designatedcoordinates Pos.

Next, the CPU 10 determines whether or not the designated positionindicated by the designated coordinates Pos is included within thescreen (step S14). Specifically, the CPU 10 determines whether or notthe front surface of the controller 7 (a side through which light to becaptured by the image capture information computing section 74 enters)is directed toward the screen. More specifically, when the designatedcoordinates Pos indicate the absence of a marker coordinate point, theCPU 10 determines that the designated position indicated by thedesignated coordinates Pos is not included within the screen, and whenotherwise, the CPU 10 determines that the designated position indicatedby the designated coordinates Pos is included within the screen.

Next, the CPU 10 sets an operation mode. Specifically, when, as a resultof the determination in step S14, the CPU 10 determines that thedesignated position indicated by the designated coordinates Pos isincluded within the screen (YES in step S14), the CPU 10 sets “0”indicating the “pointing mode” into the operation mode 128 (step S15).On the other hand, when the CPU 10 determines that the designatedposition indicated by the designated coordinates Pos is not includedwithin the screen (NO in step S14), the CPU 10 sets “1” indicating the“cross key mode” into the operation mode 128 (step S16). Thus, theinitialization process is completed.

Referring back to FIG. 17, when the initialization process is completed,the CPU 10 next executes a one-frame process (step S2). FIG. 19 is aflowchart showing the one-frame process of step S2 in detail. In FIG.19, initially, the CPU 10 executes a process of calculating thedesignated coordinates Pos (step S21) This process is similar to that ofstep S13 and will not be described.

Next, the CPU 10 determines whether or not the current operation mode isthe “pointing mode”, with reference to the operation mode 128 (stepS22). In other words, the CPU 10 determines whether or not the operationmode 128 has the value “0”. When the CPU 10 determines that theoperation mode is not the “pointing mode” (NO in step S22), the CPU 10executes a cross key mode process (step S31). This process will bedescribed below.

On the other hand, when the operation mode is the “pointing mode” (YESin step S22), then the CPU 10 determines whether or not the cross key 72a has been pressed down, with reference to the operation button data1253 (step S23). When the CPU 10 determines that the cross key 72 a hasnot been pressed down (NO in step S23), the CPU 10 executes a process ofmoving the pointer 104 to a position on the screen indicated by thedesignated coordinates Pos (step S24). In other words, the “pointingmode” process is continued. In this case, when the pointer 104 is movedto any object (i.e., any of the objects is pointed by the pointer 104),the object is selected (more specifically, a focus is set on the object,which is then in the selected stated). Moreover, the CPU 10 arranges theselection frame 105 around the selected object. Thereafter, the CPU goesto step S30 described below in which the pointer 104 is drawn at aposition where the pointer 104 has been moved (when any object isselected, the selection frame 105 is also drawn).

On other hand, when the CPU 10 determines in step S23 that the cross key72 a has been pressed down (YES in step S23), the CPU 10 executes aprocess of selecting any of the objects within the screen (step S25).Specifically, when the CPU 10 executes the determination, then if any ofthe objects has already been selected (see, for example, FIG. 11), anobject is selected that is closest to a designated position of theobjects present in the pressed direction of the cross key 72 a withreference to the position of the selected object. When the CPU 10executes the determination, then if no object has been selected (see,for example, FIG. 10), an object is selected that is closest to thecurrent designated position (a position indicated by the designatedcoordinates Pos) (when there are a plurality of closest objects, anobject present in the pressed direction of the cross key 72 a isselected). In other words, by this process, when the cross key 72 a ispressed down in the pointing mode, any of the objects is immediatelyselected.

Next, the CPU 10 moves the pointer 104 to a position in the vicinity ofa center of the selected object (an object on which a focus is set) (seeFIGS. 13 and 14) (step S26). In this case, the CPU 10 also places theselection frame 105 at the selected object.

Next, the CPU 10 sets, into the operation mode 128, “1” that is a valueindicating the cross key mode (step S27).

Next, the CPU 10 Initializes the acceleration array 127 (step S28) andinitializes the no-input time 129 (step S29).

Next, the CPU 10 executes a drawing process (step S30) Specifically, aprocess of generating an image as shown in FIG. 10 or the like, anddisplaying it as a game image on the television 2 is executed.Thereafter, the CPU 10 ends the one-frame process.

Next, a process when the CPU 10 determines in step S22 that theoperation mode is the “cross key mode” (the value of the operation mode128 is “1”) will be described. In this case, the CPU 10 executes a crosskey mode process (step S31). FIG. 20 is a flowchart showing the crosskey mode process of step S31 in detail. In FIG. 20, initially, the CPU10 determines whether or not the cross key 72 a has been pressed down,with reference to the operation button data 1253 (step S41). When theCPU 10 determines that the cross key 72 a has been pressed down (YES instep S41), a process of continuing the cross key mode process isexecuted. Initially, the CPU 10 initializes the acceleration array 127(step S42) and initializes the no-input time 129 (step S43).

Next, the CPU 10 executes a process of selecting an object, depending onthe pressed direction of the cross key 72 a (step S44). Specifically, anobject is selected that is closest of the objects present in the presseddirection of the cross key 72 a with reference to the position of thepreviously selected object. Also, as described above with reference toFIGS. 11 and 13, when the “left” direction is pressed while the leftmostobject 101 is selected within the screen, then the rightmost object 103is selected. Similarly, when the “right” direction is pressed while therightmost object 103 is selected, then the leftmost object 101 isselected.

Next, the CPU 10 moves the pointer 104 to a position in the vicinity ofa center of the selected object (see FIG. 13, etc.) (step S45). Also, inthis case, the CPU 10 places the selection frame 105 at the selectedobject. Thereafter, the CPU 10 ends the cross key mode process.

Next, a process when the CPU 10 determines in step S41 that the crosskey 72 a has not been pressed will be described (NO in step S41). Inthis case, the CPU 10 determines whether or not the controller 7 hasbeen swung or the like, and executes a process of switching the crosskey mode to the pointing mode, depending on the result of thedetermination. Specifically, initially, the CPU 10 adds “1” to theno-input time 129 (step S46). Next, the CPU 10 obtains the accelerationvector V1 by referencing the acceleration data 1251 (step S47).

Next, the CPU 10 obtains the previous acceleration vector 126 andcalculates an amount q of a change in acceleration (step S48). Theacceleration change amount q is the amount of a change from anacceleration vector calculated by a process with respect to theimmediately previous frame to an acceleration vector calculated by aprocess with respect to the current frame. The change amount q is, forexample, calculated as follows. Initially, a change vector Vs iscalculated by:

the change vector Vs=the acceleration vector V1−the previousacceleration vector V2.

The magnitude (scalar value) of the calculated change vector Vs isdefined as the change amount q.

Next, the CPU 10 updates the acceleration array (step S49).Specifically, the CPU 10 updates data of the acceleration array elementQ[N−1] with data of Q[N−2], data of Q[N−2] with data of Q[N−3], and soon. Finally, the calculated change amount q is stored into theacceleration array element Q[0].

Next, the CPU 10 calculates an integrated value Qt of the values of theacceleration array 127 (step S50) by:

the integrated value Qt=Q[0]+Q[1]+Q[2]+ . . . +Q[N−1].

Next, the previous acceleration vector 126 is updated with the value ofthe acceleration vector V1 (step S51).

Next, the CPU 10 determines whether or not the integrated value Qtcalculated in step S50 is larger than or equal to a predeterminedthreshold (step S52). Specifically, the CPU 10 determines whether or notthe amount of a change in acceleration within a predetermined time thatis indicated by the number of elements of the acceleration array 127 issufficiently large so that an operation of the controller 7 can bedetermined as “swing”. When the CPU 10 determines that the integratedvalue Qt is larger than or equal to the first threshold (YES in stepS52), it is considered that a “swing” operation has been performed, sothat the CPU 10 sets “0” (a value indicating the pointing mode) into theoperation mode 128 (step S54).

On the other hand, when the integrated value Qt is not larger than orequal to the threshold (NO in step S52), then the CPU 10 determineswhether or not the no-input time 129 is larger than or equal to apredetermined second threshold and a position indicated by thedesignated coordinates Pos is included within the screen (step S53).Specifically, the CPU 10 determines whether or not the cross key 72 ahas not been pressed down for a predetermined time or more and theplayer is pointing the screen using the controller 7. When the CPU 10determines that the aforementioned conditions are satisfied (YES in stepS53), the CPU 10 goes to step S54, and executes a process of setting theoperation mode to be the “pointing mode”. Specifically, when the playerhas not pressed the cross key 72 a for a predetermined time whilekeeping the front surface of the controller 7 directed toward the screenin the “cross key mode”, the CPU 10 switches the operation mode to thepointing mode.

On the other hand, when the CPU 10 determines in step S53 that theaforementioned conditions are not satisfied (NO in step S53), the CPU 10ends the cross key mode process. If the cross key mode process is ended,the CPU 10 goes to the process of step S30 of FIG. 19, so that theprocess of drawing the pointer 104 or the like is executed, and then theone-frame process is ended.

Referring back to FIG. 17, after the one-frame process of step S2, theCPU 10 determines in step S3 whether to end the game. When the result isYES, the CPU 10 ends the game process, and when the result is NO, theCPU 10 goes back to step S2, and then repeats the game process. Thus,the game process of this embodiment is ended.

Thus, in this embodiment, when a “swing” operation is detected during anoperation of the cross key 72 a, she operation mode is switched to thepointing mode. Also, the operation mode is switched to the cross keymode by pressing the cross key 72 a during a pointing operation. Inother words, when a motion that is directly linked to an operation to besubsequently performed (specifically, an operation of performingselection using the cross key 72 a) is desired, the cross key 72 a ispressed down. When the player desires to perform a pointing operation bymoving the pointer 104, the player swings the controller 7 itself. Theseoperations can switch the operation modes. Thereby, a plurality ofselection operation methods can be switched in quick response and withan operation free of unnaturalness.

Also, as described above, in this embodiment, change amounts ofacceleration data are accumulated in the finite acceleration array 127,and depending on whether or not the integrated value exceeds apredetermined threshold, it is determined whether or not the cross keymode is switched to the pointing mode. By thus buffering accelerationdata in a finite array, it is, for example, possible to avoid asituation that when a motion applied to the controller 7 is small but iscontinued for a long time, and therefore, acceleration data isaccumulated for a long time, so that the aforementioned switching of theoperation modes is falsely executed. In other words, if a certain levelof motion that can be considered to be a “swing” operation does notoccur for a predetermined period of time (depending on the number ofelements of the array), the operation modes are not switched, so thatthe switching of the operation modes that is not intended by the usercan be prevented from occurring. The length of the predetermined periodof time (the number of elements of the array) may be set as appropriate,depending on the game process or the like.

Although it has been assumed in the embodiment above that accelerationdata output from an acceleration sensor is used to detect a “swing”operation, the “swing” operation may be detected based on angularvelocity data output from a gyro-sensor. In this case, for example, thecontroller 7 includes a three-axis gyro-sensor. Alternatively, anextension unit including the three-axis gyro-sensor may be connected tothe controller 7. Thereafter, an angular velocity may be detected andoutput as angular velocity data, and may be used instead of or incombination with the acceleration data, thereby executing detection of a“swing” operation.

Further, a change amount of a designated position may be used instead ofdetection of a “swing” operation. Specifically, when a large changeoccurs in a designated position within a predetermined time, it may bedetermined that a large “motion” is applied to the controller 7, and theoperation mode may be switched to the pointing mode. For example, in theaforementioned cross key mode process, the designated coordinates Posmay be stored as data indicating a designated position in the previousframe in the external main memory 12, and in the next process loop, achange amount between a designated coordinate point in the previousframe and the current designated coordinate point is calculated.Thereafter, if the change amount exceeds a predetermined threshold, theoperation mode may be switched to the pointing mode. The presentinvention is not limited to the comparison with the designatedcoordinate point in the previous frame. A change amount of designatedcoordinates for a period of time corresponding to several frames toseveral tens of frames may be calculated and used to determine whetherto switch the operation modes.

Also, the present invention is not limited to the acceleration, theangular velocity, and the change amount of the designated position asdescribed above. Any method that can detect a “motion” applied to thecontroller 7 in the cross key mode may be employed. The cross key modemay be switched to the pointing mode, depending on the magnitude of themotion.

Although it has also been assumed in the embodiment above that a singlehousing (the controller 7) can be used to perform two kinds ofoperations, i.e., the cross key mode and the pointing mode, the presentinvention is not limited to this. A plurality of housings may beutilized. For example, as shown in FIG. 21, an extension controller 9 isconnected to the connector 73 of the controller 7. For example, whenplaying a game, the player holds the controller 7 with his/her righthand while holding the extension controller 9 with his/her left hand, asshown in FIG. 22. In this case, the player holds the controller 7 whiledirecting the front surface (a side through which light to be capturedby the image capture information computing section 74 enters) of thecontroller 7 held with the right hand toward the markers 8L and 8R. Theextension controller 9 includes a stick 91. It is assumed that anoperation similar to that in the cross key mode can be performed byoperating the stick 91. When the player performs an operation using thestick 91 (cross key mode), then if the player swings the controller 7held with his/her right hand, the cross key mode is then switched to thepointing mode. Also, if the player operates the stick 91 of theextension controller 9 held by his/her left hand in the pointing mode,then the pointing mode is switched to the cross key mode. Moreover, inthis case, the cross key 72 a of the controller 7 maybe effective.Specifically, the stick 91 of the extension controller 9 or the crosskey 72 a of the controller 7 is caused to be ready to be performed inthe cross key mode. If the stick 91 or the cross key 72 a is presseddown in the pointing mode, the pointing mode may be switched to thecross key mode.

Also, when a plurality of housings are utilized as described above, eachhousing may include an acceleration sensor, and the operation modes maybe switched in accordance with any housing that is swung. In theaforementioned example, for example, when the controller 7 held by theright hand is swung, the operation mode may be switched to the pointingmode, and when the extension controller 9 held by the left hand isswung, the operation mode may be switched to the cross key mode. In thiscase, acceleration data output from each controller may be associatedwith and accompanied by identification information indicating the outputsource controller.

Also, when a plurality of housings are utilized (e.g., two housings areused), each housing may include a pointing device and an accelerationsensor, and the pointing device of one of the housing for which “swing”is detected may be switched to perform an operation.

Although it has also been assumed in the embodiment above that any ofthe objects displayed on the screen is invariably selected in the crosskey mode, the present invention is not limited to this. The pointer 104may be moved by a predetermined distance, depending on a direction inwhich the cross key 72 a is pressed down. In other words, in the crosskey mode, the cross key 72 a may be used to move the pointer 104.

The selection operation in the cross key mode may be performed using asingle predetermined button instead of the cross key 72 a. For example,objects may be sequentially switched in a predetermined order every timethe operation button 72 b is pressed in the cross key mode (a focus maybe moved in a predetermined order).

The present invention is also applicable not only to game processes asdescribed above, but also to general selection operations In otherinformation processes.

While the invention has been described in detail, the foregoingdescription is in all aspects illustrative and not restrictive. It isunderstood that numerous other modifications and variations can bedevised without departing from the scope of the invention.

1. A computer-readable recording medium recording an informationprocessing program executable by a computer of an information processingapparatus for designating a position on a screen based on outputs of aplurality of input means including at least one pointing device, theprogram causing the computer to function as: an motion informationobtaining means for obtaining motion information that is informationabout a motion applied to a first input device housing itself includingthe pointing device; a movement amount calculating means for calculatinga movement amount of the first input device housing based on the motioninformation; a threshold determining means for determining whether ornot the movement amount exceeds a predetermined threshold; and a firstposition designating means for designating a position based on an outputfrom the pointing device when the threshold determining means determinesthat the movement amount exceeds the predetermined threshold.
 2. Thecomputer-readable recording medium according to claim 1, wherein thefirst input device housing further includes a non-pointing device thatis an input means that is not a pointing device, of the plurality ofinput means, and the information processing program causes the computerto further function as: a determination means for determining whether ornot an input to the non-pointing device has occurred when positiondesignation is being performed by the first position designating means;and a second position designating means for designating a position basedon an output from the non-pointing device when the determination meansdetermines that an input to the non-pointing device has occurred.
 3. Thecomputer-readable recording medium according to claim 1, wherein anon-pointing device that is an input means that is not a pointingdevice, of the plurality of input means, is included in a second inputdevice housing, and the information processing program causes thecomputer to further function as: a determination means for determiningwhether or not an input to the non-pointing device has occurred whenposition designation is being performed by the first positiondesignating means; and a second position designating means forperforming position designation based on an output from the non-pointingdevice when the determination means determines that an input to thenon-pointing device has occurred.
 4. A computer-readable recordingmedium according to claim 1, wherein the information processing programcauses the computer to further function as: a switching means forswitching a position designation mode between a first positiondesignation mode in which position destination is performed using thefirst position designating means and a second position designation modein which position designation is performed based on an output from anon-pointing device that is an input means that is not a pointingdevice, and the switching means switches the position designation modeto the first position designation mode if the movement amount exceeds apredetermined threshold when the position designation mode is the secondposition designation mode, and switches the position designation mode tothe second position designation mode if an input to the non-pointingdevice has occurred when the position designation mode is the firstposition designation mode.
 5. The computer-readable recording mediumaccording to claim 4, wherein the information processing program causesthe computer to further function as: a no-input period measuring meansfor measuring a period of time during which it is determined that thereis not an input to the non-pointing device when the position designationmode is the second position designation mode, and the switching meansswitches the position designation mode to the first position designationmode when the period of time measured by the no-input period measuringmeans exceeds a predetermined value.
 6. The computer-readable recordingmedium according to claim 1, wherein the first input device housingincludes a motion sensor for detecting a motion applied to the firstinput device housing itself, and the motion information obtaining meansobtains data output from the motion sensor as the motion information. 7.The computer-readable recording medium according to claim 1, wherein themotion information obtaining means repeatedly obtains data indicating aposition designated by the pointing device, as the motion information,and the movement amount calculating means calculates the movement amountbased on a change amount of the designated positions of the pointingdevice.
 8. The computer-readable recording medium according to claim 1,wherein the information processing program causes the computer tofurther function as: an accumulation means for accumulating the movementamounts calculated by the movement amount calculating means, and thethreshold determining means determines whether or not the cumulativevalue of the accumulation means exceeds a predetermined threshold. 9.The computer-readable recording medium according to claim 8, wherein theaccumulation means accumulates the movement amounts for a predeterminedperiod of time until a current time.
 10. The computer-readable recordingmedium according to claim 8, wherein the information processing programcauses the computer to further function as: a cumulative value resettingmeans for resetting the cumulative value when an input has occurred withrespect to an input means other than the pointing device.
 11. Thecomputer-readable recording medium according to claim 1, wherein theinformation processing program causes the computer to further functionas: a movement change amount calculating means for calculating a changeamount of the movement amounts calculated by the movement amountcalculating means, and the threshold determining means determineswhether or not the change amount calculated by the movement changeamount calculating means exceeds a predetermined threshold.
 12. Thecomputer-readable recording medium according to claim 11, wherein theinformation processing program causes the computer to further functionas: a change amount accumulating means for accumulating the changeamounts calculated by the movement change amount calculating means, andthe threshold determining means determines whether or not the cumulativechange amount of the change amount accumulating means exceeds apredetermined threshold.
 13. The computer-readable recording mediumaccording to claim 12, wherein the change amount accumulating meansaccumulates the change amounts for a predetermined period of time untila current time.
 14. The computer-readable recording medium according toclaim 12, wherein the information processing program causes the computerto further function as: a cumulative value resetting means for resettingthe cumulative change amount when an input has occurred with respect toan input means other than the pointing device.
 15. The computer-readablerecording medium according to claim 1, wherein the first input devicehousing includes an image capturing means for capturing an image of atleast one object whose image to be captured and which is located in avicinity of a display device, the first position designating meansincludes: a captured image data obtaining means for obtaining capturedimage data output from the first input device housing including theimage capturing means, and the first position designating meansdesignates a position based on an object whose image to be captured isshown in a captured image indicated by the captured image data.
 16. Acomputer-readable recording medium recording an information processingprogram executable by a computer of an information processing apparatusfor selecting one of a plurality of objects displayed on a screen, theprogram causing the computer to function as: an motion informationobtaining means for obtaining motion information that is informationabout a motion applied to a predetermined input device housing itselfused for an operation of selecting the objects; and a selection modeswitching means for switching a selection mode based on the motioninformation, between a pointing selection mode and a key selection mode,the pointing selection mode being a selection mode in which any positionon the screen can be designated, and any of the objects displayed on thescreen can be selected by designating a position where the object isdisplayed, and the key selection mode being a selection mode in whichsetting of a focused state Indicating a selected state is changed amongthe objects displayed on the screen in a predetermined order, dependingon a predetermined key input.
 17. The computer-readable recording mediumaccording to claim 16, wherein the information processing program causesthe computer to further function as: a first determination means fordetermining whether or not the predetermined key input has beenperformed when the selection mode is the pointing selection mode; and amovement amount calculating means for calculating a movement amount ofthe input device itself based on the motion information when theselection mode is the key selection mode, and the selection modeswitching means switches the selection mode to the key selection modewhen the first determination means determines that the key input hasbeen performed, and switches the selection mode to the pointingselection mode when the movement amount calculated by the movementamount calculating means exceeds a predetermined threshold.
 18. Thecomputer-readable recording medium according to claim 16, wherein theinput device housing includes a motion sensor for detecting a motionapplied to the input device housing itself, and the motion informationobtaining means obtains data output from the motion sensor as the motioninformation.
 19. The computer-readable recording medium according toclaim 16, wherein the information processing program causes the computerto further function as: an accumulation means for accumulating themovement amounts calculated by the movement change amount calculatingmeans, and the selection mode switching means switches the selectionmode to the pointing selection mode when the cumulative value of theaccumulation means exceeds a predetermined threshold.
 20. Thecomputer-readable recording medium according to claim 19, wherein theaccumulation means accumulates the movement amounts for a predeterminedperiod of time until a current time.
 21. The computer-readable recordingmedium according to claim 19, wherein the information processing programcauses the computer to further function as: a cumulative value resettingmeans for resetting the cumulative value when the first determinationmeans determines that the predetermined key input has been performed.22. The computer-readable recording medium according to claim 16,wherein the information processing program causes the computer tofurther function as: a movement change amount calculating means forcalculating a change amount of the movement amounts calculated by themovement amount calculating means, and the selection mode switchingmeans switches the selection mode to the pointing selection mode whenthe change amount calculated by the movement change amount calculatingmeans exceeds a predetermined threshold.
 23. The computer-readablerecording medium according to claim 22, wherein the informationprocessing program causes the computer to further function as: a changeamount accumulating means for accumulating the change amounts calculatedby the movement change amount calculating means, and the selection modeswitching means switches the selection mode to the pointing selectionmode when the cumulative change amount of the change amount accumulatingmeans exceeds a predetermined threshold.
 24. The computer-readablerecording medium according to claim 23, wherein the change amountaccumulating means accumulates the change amounts for a predeterminedperiod of time until a current time.
 25. The computer-readable recordingmedium according to claim 23, wherein the information processing programcauses the computer to further function as: a cumulative value resettingmeans for resetting the cumulative change amount when an input hasoccurred with respect to an input means other than the pointing device.26. The computer-readable recording medium according to claim 17,wherein the information processing program causes the computer tofurther function as: a no-input period measuring means for measuring aperiod of time during which it is determined that the key input is notbeing performed when the selection mode is the key selection mode, andthe selection mode switching means switches the selection mode to thepointing selection mode when the period of time measured by the no-inputperiod measuring means exceeds a predetermined value.
 27. An informationprocessing apparatus for designating a position on a screen based onoutputs of a plurality of input means including at least one pointingdevice, comprising: an motion information obtaining means for obtainingmotion information that is information about a motion applied to a firstinput device housing itself including the pointing device; a movementamount calculating means for calculating a movement amount of the firstinput device housing based on the motion information; a thresholddetermining means for determining whether or not the movement amountexceeds a predetermined threshold; and a first position designatingmeans for designating a position based on an output from the pointingdevice when the threshold determining means determines that the movementamount exceeds the predetermined threshold.
 28. An informationprocessing apparatus for selecting one of a plurality of objectsdisplayed on a screen, comprising: an motion information obtaining meansfor obtaining motion information that is information about a motionapplied to a predetermined input device housing itself used for anoperation of selecting the objects; and a selection mode switching meansfor switching a selection mode based on the motion information, betweena pointing selection mode and a key selection mode, the pointingselection mode being a selection mode in which any position on thescreen can be designated, and any of the objects displayed on the screencan be selected by designating a position where the object is displayed,and the key selection mode being a selection mode in which setting of afocused state indicating a selected state is changed among the objectsdisplayed on the screen in a predetermined order, depending on apredetermined key input.